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Abstract 

Helicopter Health Usage Monitoring Systems (HUMS) 
have potential for providing data to support increasing the 
service life of a dynamic mechanical component in the 
transmission of a helicopter. Data collected can demonstrate 
the HUMS condition indicator responds to a specific com- 
ponent fault with appropriate alert limits and minimal false 
alarms. Defining thresholds for specific faults requires a 
tradeoff between the sensitivity of the condition indicator 
(Cl) limit to indicate damage and the number of false alarms. 
A method using Receiver Operating Characteristic (ROC) 
curves to assess Cl performance was demonstrated using Cl 
data collected from accelerometers installed on several 
UH60 Black Hawk and AH64 Apache helicopters and an 
AH64 helicopter component test stand. Results of the 
analysis indicate ROC curves can be used to reliably assess 
the performance of commercial HUMS condition indicators 
to detect damaged gears and bearings in a helicopter trans- 
mission. 

Background 

Helicopter transmission integrity is important to helicop- 
ter safety because helicopters depend on the power train for 
propulsion, lift, and flight maneuvering. Commercial HUMS 
(Health Usage Monitoring Systems) have been developed to 
detect damaged components using vibration based methods. 
Damage in helicopter transmission components produce 
specific fault patterns in the accelerometer vibration signa- 
tures. Various vibration signature analysis methods are 
developed and implemented in the commercial HUMS to 
detect faults in bearings, gears and shafts. Condition Indica- 
tors (Cl) refer to the vibration characteristics extracted from 
these signatures and are used to reflect the health of the 
component. Numerous condition indicators are calculated 
from vibration data to characterize component health. 

HUMS have potential for both economic and safety bene- 
fits. If component usage (cycles, load) is more severe than 
the design life of a component, HUMS can provide safety 
benefits. If component health has degraded due to anomalies 


caused by harsh operational environments, limited mainte- 
nance, assembly errors, or manufacturing faults, HUMS can 
also provide safety benefits. If component usage is less 
severe, and no anomalies are detected by the HUMS system, 
the service life (time between component replacement or 
overhauls) of components may be extended. In order to 
identify anomalies/faults that occur in the field within a 
specific component, the Cl must demonstrate a high level of 
reliability to detect anomalies/faults with minimal false 
alarms. 

In 1999, the FAA published Advisory Circular (AC) 29- 
2C, Section MG-15, referred to as the HUMS AC to provide 
guidance for achieving airworthiness approval for a full 
range of HUMS applications (ref. 1). In order to receive 
maintenance credits to increase the service life of a specific 
component, the HUMS application for which credits are 
sought must be validated. Evidence must be provided in the 
form of seeded fault tests and/or fielded data that demon- 
strate the HUMS respond to the specific component fault 
with appropriate alert limits. Reference 2 provides an exam- 
ple of the process for obtaining maintenance credits of a 
helicopter transmission component. 

Threshold Assessment 

Identifying a Cl that reliably detects damage to specific 
components is required for the credit validation phase of 
HUMS airworthiness approval. In order to evaluate the 
performance of an individual Cl, a threshold must be de- 
fined. Determining reliable Cl threshold limits that differen- 
tiate between healthy and faulted components is necessary to 
ensure condition indicators effectiveness. When defining the 
threshold limit of a diagnostic tool, there is a tradeoff be- 
tween the sensitivity of the limit to indicate damage and the 
number of false alarms. If a limit is decreased, damage may 
be detected, but more false alarms may result. If a limit is 
increased, false alarms may decrease, but the Cl will be less 
sensitive to damage. 

The simplest approach to set thresholds for vibration di- 
agnostic tools is to gather baseline data under normal operat- 
ing conditions, and set the threshold to values that exceed 
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normal operating conditions. This works well until a fault 
occurs and the Cl value is within the normal variance for 
that Cl across a fleet of helicopters. HUMS manufacturers 
have observed significant variances of Cl indicator levels 
between healthy gearbox components (ref. 3). 

Due to limited damage data on aircraft, diagnostic tools 
are developed in controlled ground test environments with 
seeded fault tests. Unfortunately, the response of the Cl to a 
component fault in a helicopter cannot be validated until the 
fault occurs on aircraft. This is due to the numerous vari- 
ables on aircraft that cannot be controlled or measured. The 
U.S. Army has undertaken a Condition Based Maintenance 
(CBM) program installing HUMS on fleets of helicopters. 
Some faults have occurred in the field when HUMS Cl data 
has been recorded. This data will be used to validate a 
method of defining thresholds for transmission diagnostic 
algorithms that provides the minimum number of false 
alarms while maintaining sensitivity to transmission damage, 
required for HUMS performance assessment. 

Receiver Operating Characteristic (ROC) curves will be 
used as a tool to assess the performance of different thresh- 
old values for identifying specific faults. The objective of 
this research is to demonstrate ROC curves developed from 
signal detection theory are useful analysis tools for compar- 
ing the response and performance of a Cl from a healthy 
component to a component with a specific fault by differen- 
tiating between Cl values for healthy and faulted compo- 
nents. Cl data collected from accelerometers installed on 
several UH60 Black Hawk and AH64 Apache helicopters 
collected from two onboard commercial HUMS will be 
compared using ROC curves. ROC analysis will also be 
applied to test stand data from a commercial HUMS in- 
stalled on an AH64 helicopter component test stand. Details 
on the ROC analysis will be discussed in the next section. 


Signal Detection Theory 

Signal detection theory is an analysis technique for detect- 
ing a signal in the presence of noise. Analysis techniques can 
be used to improve decision making in fields where it is a 
challenge to discriminate between detection of an anomaly 
and a false alarm. There are two main components used 
when applying information theory to a system, information 
acquisition and criterion. Information acquisition refers to 
the data, for example Cl data, available for making the 
decision. Criterion refers to the significant factor, for exam- 
ple threshold values when damage occurs, used to analyze 
the data (ref. 4). The criterion is a cutoff value used to detect 
the signal. If the Cl value is larger than the criterion value, 
the fault is deemed present. If the Cl value is less than the 
criterion, the component is deemed healthy. 

The goal in signal detection theory is to decrease the over- 
lap between the probability curves for healthy (noise signal) 
and faulted (damage signal present within the noise) condi- 


Criterion response 



Figure 1. — Probability curves for noise and signal + noise. 


tions. Statistical distributions of the healthy and faulted Cl 
data are used to generate ROC curves. ROC curves are 
plotted with the false alarm rate (probability of false alarm 
or false positive rate) on the horizontal axis (x) versus the hit 
rate (probability of detection-true alarm or true positive rate) 
on the vertical axis (y). The ROC curves can be used to 
evaluate thresholds since they provide a visual comparison 
of two or more tests on common scales at all possible 
thresholds independent of the test scale. 

Figure 1 provides a visual graph of two normal distribu- 
tions for healthy and fault data for a CL The no damage 
distribution is labeled “noise,” the damaged distribution is 
labeled “signal + noise”. The criterion line separates the 
graph into correct rejections, misses, false alarms and hits. 
The goal in signal detection theory is to decrease the noise 
and increase the signal strength to decrease the overlap 
between the two probability curves. The ability to discrimi- 
nate between the healthy and faulted conditions is repre- 
sented by the Discriminability Index ( d '). The 
discriminability index can be used to test the ability of a Cl 
to discriminate between healthy and faulted components. 
CI’s with higher d’ values will identify clear decision 
thresholds. 

d’ = separation/spread = | z (hit rate) - z (false alarm) | (1) 

The z refers to the z or standard score, the number of 
standard deviations that a given value of x is above or below 
the mean. The mean of the sample is equal to: 



n 


The standard deviation of the sample is equal to: 


s = 


XT ~ x ) 2 

n - 1 


( 3 ) 
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Variance is equal to s squared: 


Test statistic z for independent and large samples is calcu- 
lated as: 


Separation corresponds to the difference between the means. 
The spread corresponds to the standard deviation of the 
probability densities. A larger d ’ value indicates better 
performance of the Cl to detect healthy and faulted compo- 
nents. 

ROC curves are used in signal detection theory to identify 
tradeoffs between hit rates and false alarm rates. They are 
used to distinguish between a signal and noise. They are 
used in medical fields for health decision making and to 
assess the predictive accuracy of the tools used to make 
these decisions (refs. 5 and 6). Interpretation of medical 
diagnostic tests can vary between diagnosticians. The ROC 
can be used as a tool to assess the performance of a test 
independent of the threshold, providing a common metric for 
comparison (ref. 7). 

Interpretation of a signal in the presence of noise in a 
medical diagnostic test is comparable to predicting transmis- 
sion faults based on symptoms observed in the vibration 
signatures. A fault produces a change in the Cl levels that 
are compared to healthy Cl levels. ROC curves can be used 
to summarize the accuracy of the Cl to identify a specific 
fault. The ROC curves provide various alternatives as the 
criterion is moved to different levels. 

The Cl thresholds identified with the maximum detection 
capability and minimum false alarms can be integrated into 
health indicators. Health indicators (HI) are under develop- 
ment by several HUMS manufacturers (ref. 8). Health indica- 
tors provide decision making tools for the end user on the 
status of system health. Health indicators consist of the 
integration of several condition indicators into one tool that 
provides the health status of the component to the end user 
such as O.K., or needs maintenance. Cl and HI performance 
and reliability has been difficult to evaluate due to limited 
failure data available to assess performance. 

The advantage of using ROC curves as compared to sepa- 
rately viewing false alarm and damage detection rates is that 
the Cl threshold values can be compared visually for several 
faulted components at all threshold values. From this infor- 
mation you can select the Cl value with the optimal per- 
formance based on rates of false alarms and damage 
detection (refs. 9 and 10). ROC curves can be used to: 1) 
determine if a Cl can discriminate between healthy and 
faulted component states; 2) identify Cl sensitivity to dam- 
age and false alarm rate for a specific threshold value and 
fault; 3) quantify the effectiveness of a Cl to classify a 
specific fault. 


Helicopter Test Stand 



Figure 2. — Block diagram of CI/HI calculation. 


Prior to identifying the steps required to generate an ROC 
curve for a Cl, a Cl must be defined to monitor a specific 
transmission component on a helicopter. Figure 2 provides a 
block diagram of this process to calculate a Cl for a specific 
component fault. Data is collected from a helicopter or test 
stand using a sensor mounted at a location sensitive to the 
frequencies under investigation. Location and mounting can 
be optimized to obtain the best response, although installa- 
tion is often limited to space availability on the helicopter. 
HUMS installation locations are correlated with components 
under investigation. The data acquisition system for signal 
processing is sampled at speeds that provide sufficient 
vibration data for calculating asynchronous and time syn- 
chronous averaged data based on shaft speed of components 
under investigation. If the Cl is sensitive to environmental 
conditions, parameters such as torque and speed must be 
controlled or measured while maintaining steady flight 
regimes. Based on the physical phenomena of the rotating 
component (gears, bearings, shafts) interacting with its 
environment, specific fault patterns are produced in the 
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accelerometer vibration signatures for the component fault 
under investigation. The Cl used is based on its sensitivity 
for detecting specific component faults based on analytical 
and experimental data. 

Applying signal detection theory and the development of 
ROC curves requires several steps to be completed. An 
example of this process will be outlined. A flowchart of this 
process is shown in figure 3. The first step is to identify a 
specific component, fault and Cl for analysis. The com- 
ponent and Cl are selected if the component had a specific 
anomaly in the field and the Cl responded. A Cl is also 
selected if it is insensitive to operating conditions. Prelimi- 
nary review of the historical monitored data should show 
some different responses between the healthy and faulted Cl 
levels. Step two is to calculate the Cl mean and standard 
deviation for healthy/faulted component. From these 
values, two distributions for a healthy and faulted dataset are 
plotted. For this demonstration, normal distributions will be 
used. Alternative distributions can be used for each CL 
Empirical data can also be used to generate probability 
distributions via histograms scaled so that the area under the 
curve is equal to 1 . 

If significant overlap occurred in these two plots, this Cl 
would not be a good choice for differentiating between a 
healthy and faulted component. Step 3 is to identify the 
thresholds to use to generate ROC curves by reviewing the 
x axis of the distribution plots to calculate the probability of 
detection (hits) and probability of false alarms. Step 4, using 
the hit rate and false alarm rate, a ROC curve was generated. 

In the following section, ROC curves will be developed 
using condition indicators measured on several UH60 Black 
Hawk and AH64 Apache helicopters with both healthy and 
faulted components. A schematic of the Black Hawk gear- 
box is shown in appendix A. A description of the Cl used for 
indicating the anomalies and photos of the observed damage 
will be discussed. ROC analysis will also be applied to test 
stand data from the University of South Carolina AH64 
helicopter component test stand. A summary of the datasets 
for analysis are shown in table 1 . Although all of the datasets 
were analyzed, ROC curves were only generated for the 
highlighted datasets. In order to perform a thorough analysis 
of this tool, four Cl datasets are required: healthy helicopter 
component, faulted helicopter component, healthy test stand 
component, faulted test stand component. To date, four 
complete datasets are not available for analysis. For this 
reason, the validation is limited to the steps required to 
perform this analysis on the available datasets. 



Figure 3. — Flowchart of ROC analysis. 
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TABLE 1.— SUMMARY OF DATASETS 


Dataset timeframe 

Component fault type 

Cl 

Sensor 

Goodrich IVHMS 




UH60 Tail 9326485 - Healthy 
2/05-6/07 

NA - Baseline 

RK - lOOt main bevel, 
Bearing Energies 

Main Bevel left/right 

UH60 Tail 9326507 - Faulted 
9/06-6/07 
I AC VMEP 

Main Bevel Gear Coating Anomaly 

RK - 1 OOt main bevel 

Main Bevel left/right 

UH60 83-23913 

Tail Gearbox Output Gear Scoring 

FM4 - Tail gearbox 

Tail Gearbox 19t 

2/04-9/06 


output gear 

input 

UH60 83-23874 

Oil cooler fan bearings pitting on race and 

Bearing Energy 

Oil Cooler Bearing 

2/04-12/06 

cages, spalling and pitting on balls 


Energy 

UH60 83-23900 

Oil cooler fan bearings pitting on race and 

Bearing Energy 

Oil Cooler Bearing 

1/04-9/07 

cages, spalling and pitting on balls 


Energy 

UH60 (83-23911, 83-23845, 83-23914, 83-23921) 
2/04-11/07 

NA - Healthy 

Bearing Energy 

Oil Cooler Bearing 
Energy 

AH64A 88-00213 -Faulted 

Nose gearbox input roller bearing spalling 

Input/Output Bearing 

Nose gearbox#2 

10/05-5/07 

on outer race and 5 rollers and cage wear 

Energy 

input bearing 

AH64D 01-05270 - Faulted 
9/04-1/05 

Aft hanger bearing spalling and corrosion 
pitting of single ball 

Bearing Energy 

Aft hanger bearing 

AH64D (04-05462, 04-05464, 04-05479, 05-07005, 
05-07007, 05-07009) 

9/06-1/08 

I AC VMEP - USC-64D-Test Stand 

NA - Healthy 

Fwd/Aft HB Energy 

Fwd/Aft hanger 
bearing 

USC-64D TB-0001, TB-0002* 
10/07 

Aft Hanger Bearing - low grease 

Aft HB energy 

Aft hanger bearing 

USC-64D TB-0003* 
10/07-1/08 

Aft Hanger Bearing - coarse sand con- 
taminated 

Aft HB energy 

Aft hanger bearing 

USC-64D TB-0001, 0002, 0003* 
10/07-1/08 

Fwd Hanger Bearing - seeded fault, 3 
small pits on inner race 

Fwd HB Energy 

Fwd hanger bearing 


* Seeded fault 


Data Analysis 


Gears 

UH60 Main Gearbox Bevel Gear Anomaly 

Cl data was collected from an onboard commercial 
HUMS system, Integrated Vehicle Health Management 
System (IVHMS), from a fleet of Black Hawk helicopters 
(ref. 11). A gear anomaly occurred on one helicopter, as 
shown in figure 4. Residual Kurtosis (RK), a gear Cl, was 
plotted from the accelerometer located on the main bevel 
gear right and left output flange. The Cl, referred to as RK in 
the IVHMS, is comparable to a more common gear Cl 
Figure of Merit 4 (FM4). In addition, RK was also plotted 
for another helicopter without an anomaly. 

FM4 was developed to detect changes in the vibration 
pattern due to damage (pitting, small, cracks) on a limited 
number of teeth (ref. 12). FM4 is calculated by sampling the 
accelerometer data and time synchronous averaging the data 
to shaft speed, converting the data to frequency domain, 
removing the gear meshing frequency, harmonics, and 1st 
order sidebands from the original signal, converting it back 
to the time domain, then calculating the fourth normalized 
statistical moment (normalized kurtosis) of the signal. When 
one or two teeth develop a defect (such as a crack or pitting) 
a peak or series of peaks appears in the difference signal and 



Figure 4.— UH60 main bevel gear. 
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FM4 reacts by increasing above this nominal value of 3. A 
block diagram of this calculation can be found in the appen- 
dix. Details of this Cl can be found in reference (ref. 8). 

Figures 5 and 6 are plots of the RK values for acceler- 
ometers installed on the main bevel gear right and left 
modules on both helicopters. The data is plotted versus 
collected data points (readings) for that helicopter. Note that 
the RK values for the helicopter with the gear anomaly (tail 
507) are higher than the RK values measured from the 
helicopter with the healthy gear (tail 485). Also note RK 
values measured from the left accelerometer for tail 507 are 
higher than those measured from the right accelerometer. 

Figure 7 shows the distribution of the RK values for the 
two accelerometers installed on the two helicopters. There is 
no overlap of fault and no fault data above a RK value of 4, 
and minimal overlap above 3.5 for the right accelerometer. 
Based on the mean and distribution of these two accelerome- 
ters installed on the two helicopters, the ROC curve was 
created and is shown in figure 8 and the data used to generate 
the ROC curve is shown in table 2. Based on this small 
dataset, values above 4 would result in no false alarms for this 
anomaly using Cl residual kurtosis. Referring to table 2, a 
threshold value of 4 has the largest discriminability index. 


TABLE 2.— ROC CURVE DATA 
FOR RK RIGHT MODULE 


RK 

threshold 

Hit 

rate 

z (hit) 

False 

alarms 

z (FA) 

dC 

3 

0.96 

-1.8 

0.27 

0.6 

2.4 

3.5 

0.84 

-1 

0.01 

2.4 

3.4 

4 

0.58 

-0.2 

0 

4 

4.2 

4.5 

0.21 

0.8 

0 

4 

3.2 



Tail 507 left RK 
Tail 485 left RK 




0 500 1000 1500 2000 2500 

Reading 

Figure 5. — Residual kurtosis for left module. 



Figure 6 — Residual kurtosis for right module. 
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UH60 Tail Gearbox Output Gear Anomaly 

Cl data was collected from an onboard commercial HUMS 
system, Vibration Management Enhancement Program 
(VMEP) on several Black Hawk helicopters (ref. 13). A tail 
gearbox was removed from service due to high vibration 
measured by the VMEP system. Diametral scoring (fig. 9) 
was observed on the top land near the toe of the tail gearbox 
output gear. In addition, a non-magnetic wire, approximately 
3 -in. long, was found in the outboard (seal side) of the input 
bearing. The wire was protruding about 14 in. between the 
roller and cage near the smaller diameter and was wedged 
behind the rollers. 

Figure 10 is a plot of the FM4 values before and after re- 
placement. After the tail gearbox was removed and replaced, 
the measured vibration decreased to below 3. Due to the 
limited amount of data, distributions required for generating 
ROC curves could not be created for this dataset. However, if 
you look at the FM4 values before and after replacement, all 
of the values prior to replacement were at values above 4. The 


values were under 3 when the gearbox was replaced. There is 
no overlap between the damaged and healthy values. A 
threshold of 3.5 could also be used for this geared component. 

Bearings 

UH60 Oil Cooler Fan Bearings 

Cl data was collected from VMEP on two helicopters 
when the HUMS indicated a problem with the oil coolers at 
initial installation that remained for 8 months. Both oil 
coolers were installed in Black Hawk helicopters in the same 
unit and were replaced. Damage observed in the oil coolers 
after replacement included pitting on the races, cages, and 
balls of the oil cooler fan bearings. Pitting, spalling and 
deformation of balls were also observed. Damage to the race 
and balls is shown in figure 1 1 . 

Bearing Energy was the Cl used to detect the oil cooler 
fan bearing fault. Bearing Energy is calculated as the root- 
sum-square (RSS) of the asynchronous vibration spectrum 
filtered around select frequency bands for specific bearings 
(ref. 13). Bearing Energy Cl for the two helicopters is shown 
in figure 12, both before and after bearing replacement noted 
by the green diamond on the plot. Values exceeded 6 during 
time period when the components were damaged and 
dropped to below 4 when the bearings were replaced. 

Preliminary review of the historical monitored data 
should show some different responses between the healthy 
and faulted Cl levels. In order to verify the Cl would not 
respond to healthy conditions, four helicopters in the same 
unit that did not experience a fault were analyzed. The oil 
cooler fan bearing energy Cl for these four helicopters is 
shown in figure 13. Most values are less than the minimum 
faulted values, but there is some overlap with one helicopter. 

The mean and standard deviation was then calculated for 
the Bearing Energy values when damage was present and for 
the four healthy helicopters. From these values two distribu- 
tions for a healthy and faulted dataset were plotted and are 
shown in figure 14. If significant overlap occurred in these 
two plots, this Cl would not be a good choice for differenti- 
ating between a healthy and faulted component. 



Figure 9. — Scoring damage on UH60 tail gearbox output gear. 
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Figure 10. — FM4 values for tail gearbox. 




Figure 11 

.—Oil cooler fan bearing damage. 
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Figure 12,— Bearing energy values for oil cooler fan 
bearings. 




Threshold values used to generate ROC curves were iden- 
tified by reviewing the x axis of the distribution plots. 
Values of 5, 6, 7, and 8 were selected. From the normal 
distributions and thresholds, false alarms and hit rates were 
calculated for bearing energy threshold values. Using the hit 
rate and false alarm rate, a ROC curve was plotted for this 
dataset and is shown in figure 15. This ROC curve shows the 
tradeoff between hit rates and false alarm rates based on the 
4 thresholds listed in Table 3. 


TABLE 3.— ROC CURVE DATA 
FOR BEARING ENERGY 


Bearing energy 
threshold 

Hit rate 

z (Hit) 

FA 

alarms 

z (FA) 

d 

5 

1.00 

-3.58 

0.14 

1.09 

4.66 

6 

0.99 

-2.32 

0.03 

1.88 

4.20 

7 

0.86 

-1.06 

0.00 

2.68 

3.74 

8 

0.42 

0.19 

0.00 

3.47 

3.28 
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AH64A Nose Gearbox Bearing 

Cl data was collected from VMEP on a helicopter with a 
damaged nose gearbox input roller bearing. Spalling on the 
outer race, spalling on 5 rollers and cage wear was observed 
on the bearing. Pitting and scoring on splines of quill shaft 
were also observed. Damage to the outer race is shown in 
figure 16. 

Bearing Energy Values for Nose Gearbox #2 Input Bear- 
ing are shown in figure 17. Values exceeded 6 during time 
period when the components were damaged and dropped to 
below 6 when the bearings were replaced. Due to the limited 
amount of data, distributions required for generating ROC 
curves could not be created for this dataset. However, if you 
look at the Bearing Energy values before and after replace- 
ment, all of the values prior to replacement were at values 
above 20. The values were under 7 when the gearbox was 
replaced. There is no overlap between the damaged and 
healthy values. A threshold of 7 could also be used for this 
geared component. 



Figure 16. — Damage to the outer race. 



Figure 17.— Bearing energy values for nose gearbox 2 
input bearing. 


AH64D Hanger Bearing 

Cl data was collected from a VMEP equipped helicopter 
when a hanger bearing was severely spalled. The bearing 
was removed and a new bearing reinstalled. At removal it 
was found that the failure was initiated by spalling and 
pitting of a single ball, which then caused secondary damage 
to the other balls and races. Bearing grease contaminated 
with dirt and damage to the ball is shown in figure 18. 



Figure 18. — Bearing grease contaminated with dirt and 
damage to the ball. 
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Figure 19. — Bearing energy values for aft hanger bearing. 


Bearing Energy for the aft hanger bearing is shown in 
figure 19, both before and after bearing replacement. Values 
exceeded 6 during the time period when the components 
were damaged and dropped to below 3 when the bearings 
were replaced. Due to the limited amount of data, distribu- 
tions required for generating ROC curves could not be 
created for this dataset. 

AH64D Test Stand Hanger Bearing 

Tests were performed in the University of South Carolina 
AH64D test stand with hanger bearing faults (ref. 14). Two 
aft hanger bearings were tested with low grease and one aft 
hanger bearing was tested with grease contaminated with 
coarse sand. During all three tests, a forward hanger bearing 
with three 0.030-in. pits milled on the inner race was also 
installed in the test rig. The purpose of the milled pits was an 
attempt to generate additional spalling in the bearing over 
the course of the test. 

Bearing Energy was the Cl used to detect the faults in the 
forward and aft hanger bearings. Figure 20 is a plot of the aft 
hanger bearing energy values during the two low grease 
tests. Figure 21 is a plot of the aft hanger bearing energy 
values during the grease contaminated test. Figure 22 is a 
plot of the forward hanger bearing energy values measured 
during the three tests with the pitted bearing inner race. 

In order to verify the Cl response to healthy conditions, 
six helicopters from the same unit that did not experience 
tail rotor drive shaft anomalies were analyzed. The forward 
and aft hanger bearing energy values are plotted in 
figures 23 and 24 for the six helicopters. The aft bearing 
energy values for the low grease and the six helicopters were 
very similar, no significant differences were observed 
between Cl values. But, when compared to the bearing 
contaminated with sand, significant differences were ob- 
served between the six healthy helicopters and the bearing 
contaminated with sand. The forward hanger bearing with 
the inner race pits is slowly increasing, but does not look 
significantly different than the forward hanger bearings from 
the six healthy helicopters. 
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Figure 20.— Bearing energy values for low grease aft 
hanger bearings. 
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Figure 21.— Bearing energy values for coarse sand aft 
hanger bearing. 
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Figure 22 —Bearing energy values for pitted forward 
hanger bearing. 
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Figure 23. — Bearing energy values for aft hanger bearing. 
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Figure 24. — Bearing energy values for forward hanger 
bearing. 
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Figure 26.— Forward hanger bearing-healthy helicopter, 
test stand pitted inner race. 


The mean and standard deviation was first calculated for 
the Cl Bearing Energy values of the six healthy test stand aft 
hanger bearings and the test stand aft hanger bearings with 
contaminated grease. From these values two distributions for 


a healthy and faulted dataset were plotted and are shown in 
figure 25. No overlap occurred in these two plots indicating 
this Cl is a good choice for this type of fault. Comparing the 
bearing energy values for the healthy helicopter transmission 
and the faulted test stand, thresholds set at 5, 6, 7, and 8, 
used in detecting previously discussed bearing anomalies, 
result in 100% detection rate and 0% false alarm rate. 
Although enough data points were not stored when this same 
fault occurred in flight (see fig. 19) to generate a distribu- 
tion, the mean of the dataset when damage occurred was 
equal to 29.56, and after replacement was 1.87 as compared 
to the damaged test stand mean, 18.214, and healthy helicop- 
ter mean, 1.3374. 

The mean and standard deviation was then calculated for 
the Bearing Energy values of the six healthy test stand fwd 
hanger bearings and the test stand fwd hanger bearings with 
the pitted inner race. From these values two distributions for 
a healthy and faulted dataset were plotted and are shown in 
figure 26. Significant overlap occurred in these two plots, 
indicating this Cl would not be a good choice for differenti- 
ating between a healthy and faulted component. 


Summary/Conclusions 

The objective of this research was to demonstrate ROC 
curves, which are used to compare Cl response between 
healthy and faulted components, can be used to: 1) deter- 
mine if a Cl can discriminate between healthy and faulted 
component; 2) identify Cl sensitivity to damage and false 
alarm rate for a specific threshold value; 3) quantify the 
effectiveness of a Cl to classify a specific fault. 

Cl data was analyzed from accelerometers installed on 
several UH60 Black Hawk and AH64 Apache helicopters 
collected from two onboard commercial HUMS. Data was 
also analyzed from a commercial HUMS installed on an 
AH64 test stand. 

Results of the analysis indicate ROC curves can be used to 
assess the performance of commercial HUMS CL Four Cl 
datasets are required: healthy helicopter component, faulted 
helicopter component, healthy test stand component, faulted test 
stand component. To date, four complete datasets were not 
available. As confirmed faults are detected on HUMS equipped 
helicopters this analysis will continue to be validated. 

Additional field data is also required to assess Cl per- 
formance on different faulted components. The ROC curves 
can determine the probability of detection and false alarms 
based on a Cl threshold. The ROC curves can be incorpo- 
rated into health indicators that provide decision making 
tools for the end user on component health. 
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Appendix A 



Figure A.1 . — Black Hawk gearbox schematic (3400 Series) [ref. 15], 
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Appendix B 


Time synchronous averaged A Input 

(TSA) vibration data N sensor data 

V' 


Fast Fourier transform 

V 

FM4 — gear meshing 
frequency harmonic, 
first order sidebands 

r:ii Ar 

Filter 

<> 





inverse fast Fourier transform 



o 


Difference signal (filtered ISA) = 

d 


, N 

~ Z {d n -df 

N n = 1 


FM4 


A N 

77 Z (d n -d) 2 

_ N n= 1 J 

2 


Output 

Cl 


Figure B.l. — FM4 calculation. 
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